Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

⬆️ Upgrade the go.etcd.io/etcd dependency to v3.4.34 #795

Merged
merged 5 commits into from
Nov 8, 2024
Merged

⬆️ Upgrade the go.etcd.io/etcd dependency to v3.4.34 #795

merged 5 commits into from
Nov 8, 2024

Conversation

renormalize
Copy link
Member

@renormalize renormalize commented Oct 21, 2024 •
edited
Loading

What this PR does / why we need it:

  • Upgrade the etcd version to v3.4.34.
  • Use the Restoration API provided by etcd to trigger a restore instead of relying on copied code for restoration.
  • Upgrade the etcd version to v3.4.34 in the integration tests.

Which issue(s) this PR fixes:
Fixes #792

Special notes for your reviewer:

Only commits beb9cbc and b849aaa have code changes which are relevant.

cc @seshachalam-yv

Release note:

`etcd-backup-restore` now uses etcd version `v3.4.34` for the embedded etcd during restoration.

`etcd-backup-restore` now uses the `Restore` API exposed by `go.etcd.io/etcd/clientv3/snapshot` to perform restoration of the etcd data directory, moving away from performing restoration of the data directory manually.

@renormalize renormalize added kind/enhancement Enhancement, improvement, extension area/dev-productivity Developer productivity related (how to improve development) kind/cleanup Something that is not needed anymore and can be cleaned up platform/all area/performance Performance (across all domains, such as control plane, networking, storage, etc.) related area/control-plane Control plane related dependencies Pull requests that update a dependency file labels Oct 21, 2024
@renormalize renormalize added this to the 2024-Q4 milestone Oct 21, 2024
@renormalize renormalize self-assigned this Oct 21, 2024
@renormalize renormalize requested a review from a team as a code owner October 21, 2024 08:16
@gardener-robot gardener-robot added the needs/review Needs review label Oct 21, 2024
@gardener-robot-ci-1 gardener-robot-ci-1 added reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) needs/ok-to-test Needs approval for testing (check PR in detail before setting this label because PR is run on CI/CD) and removed reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) labels Oct 21, 2024
@gardener-robot gardener-robot added size/xl Size of pull request is huge (see gardener-robot robot/bots/size.py) needs/second-opinion Needs second review by someone else labels Oct 21, 2024
@gardener-robot-ci-2 gardener-robot-ci-2 added reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) and removed reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) labels Oct 21, 2024
seshachalam-yv
seshachalam-yv requested changes Oct 21, 2024
View reviewed changes
pkg/snapshot/restorer/restorer.go Outdated Show resolved Hide resolved
pkg/snapshot/restorer/restorer.go Outdated Show resolved Hide resolved
@gardener-robot gardener-robot added the needs/changes Needs (more) changes label Oct 21, 2024
@gardener-robot-ci-2 gardener-robot-ci-2 added reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) and removed reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) labels Nov 5, 2024
@gardener-robot-ci-2 gardener-robot-ci-2 added the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Nov 5, 2024
@gardener-robot-ci-1 gardener-robot-ci-1 removed the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Nov 5, 2024
@gardener-robot-ci-3 gardener-robot-ci-3 added the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Nov 5, 2024
@gardener-robot-ci-1 gardener-robot-ci-1 removed the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Nov 5, 2024
@gardener-robot-ci-1 gardener-robot-ci-1 added the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Nov 6, 2024
@renormalize renormalize changed the title ⬆️ Upgrade the go.etcd.io/etcd dependency to v3.4.34, and all other direct dependencies ⬆️ Upgrade the go.etcd.io/etcd dependency to v3.4.34 Nov 6, 2024
@gardener-robot-ci-1 gardener-robot-ci-1 removed the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Nov 6, 2024
renormalize and others added 4 commits November 6, 2024 15:54
@renormalize
Upgrade go.etcd.io/etcd to v3.4.34.
8faea3f 
* Upgrade `go.etcd.io/etcd` to `v3.4.34`.

* Remove all `replace` directives which locked versions of dependencies.
@renormalize @seshachalam-yv
Use the Restore API in clientv3/snapshot for etcd DB restoration.
beb9cbc 
* Use the `Restore` API provided by the `clientv3/snapshot` package,
  and remove copied code for performing restoration of the etcd database
  from the base snapshot.

* The temporary directory to persist full and delta snapshots in created
  at the start of the `Restore()` call. This was created only before
  deltas were fetched previously.

* Base full snapshot is also now stored in the `TempSnapshotDir`.

* It is now assumed that the etcd data directory, which is typically
  `default.etcd` itself is not present before restoration, and not just
  the `member` directory.

* Unit tests are adapted accordingly such that the etcd data directory
  is removed before restoration.

---------

Co-authored-by: Seshachalam Yerasala Venkata <[email protected]>
@renormalize
Compactor does not create the database directory anymore, since the R…
b849aaa 
…estore API does it.

* Restoration through the `Restore` API will fail if a directory for the database is
  already present. Due to this, explicitly creating a directory before restoration is
  triggerred is not performed anymore. The `Restore` call creates the directory.
  Cleanup of this directory is still performed, as a deferred call.

* Fix broken unit tests in `pkg/compactor`.
@renormalize
Upgrade the etcd version to v3.4.34 in the integration tests.
90087a7
@gardener-robot-ci-2 gardener-robot-ci-2 added reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) and removed reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) labels Nov 6, 2024
@seshachalam-yv
Copy link
Contributor

seshachalam-yv commented Nov 6, 2024 •
edited
Loading

It appears that we are currently referencing etcd version 3.4.13 in the following files:

Could you please update these references to etcd version 3.4.34?

seshachalam-yv
seshachalam-yv reviewed Nov 6, 2024
View reviewed changes
Copy link
Contributor

@seshachalam-yv seshachalam-yv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall, the changes look good to me. Thank you, @renormalize, for upgrading the etcd version.

@renormalize
Copy link
Member Author

Thanks for your review @seshachalam-yv!

Regarding your comments:

@renormalize
Replace references to etcd-custom-image with etcd-wrapper.
5cc945f 
* References to etcd-custom-image with version `v3.4.13-bootstrap-1` are now
  replaced with references to etcd-wrapper with version `v0.2.0`.
@gardener-robot-ci-2 gardener-robot-ci-2 added the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Nov 7, 2024
@gardener-robot-ci-3 gardener-robot-ci-3 removed the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Nov 7, 2024
@renormalize
Copy link
Member Author

@seshachalam-yv

I've run etcd-druid e2e tests with changes proposed from this PR, along with the changes proposed in gardener/etcd-wrapper#30.

I overrode the etcd-backup-restore dependency in etcd-druid with the latest commit of this PR.

All tests pass.

make ci-e2e-kind 
➜  etcd-druid git:(etcd-v3.4.34) ✗ make ci-e2e-kind                       
Creating local docker registry...
bdc16e1971652fe76195d22d055913be04bee78d99fc72ddbc53ead32f85d1f3
Generating kind cluster config...
Creating cluster "etcd-druid-e2e" ...
 ✓ Ensuring node image (kindest/node:v1.29.8) 🖼
 ✓ Preparing nodes 📦  
 ✓ Writing configuration 📜 
 ✓ Starting control-plane 🕹️ 
 ✓ Installing CNI 🔌 
 ✓ Installing StorageClass 💾 
Set kubectl context to "kind-etcd-druid-e2e"
You can now use your cluster with:

kubectl cluster-info --context kind-etcd-druid-e2e

Have a nice day! 👋
Initializing local docker registry...
configmap/local-registry-hosting created

📌 NOTE: To target the newly created KinD cluster, please run the following command:

    export KUBECONFIG=/Users/renormalize/go/src/github.com/gardener/etcd-druid/hack/kind/kubeconfig

node/etcd-druid-e2e-control-plane condition met
deployment.apps/localstack created
service/localstack created
Waiting for deployment "localstack" rollout to finish: 0 of 1 updated replicas are available...
deployment "localstack" successfully rolled out
pod/localstack-7f769fd7dc-p6j5c condition met
namespace/e2e-test created
-------------------
Setting up environment
-------------------
namespace/e2e-test unchanged
No tags generated
Starting test...
Starting deploy...
Starting pre-deploy hooks...
Deleting previous job
No resources found
Completed pre-deploy hooks
 - configmap/aws-infra-common created
 - configmap/aws-run-script created
 - secret/aws-infra-secret created
 - job.batch/aws-infra-job created
Starting post-deploy hooks...
Wait until job is ready
job.batch/aws-infra-job condition met
Completed post-deploy hooks
You can also run [skaffold run --tail] to get the logs

-------------------
Deploying Druid
-------------------
Generating tags...
 - europe-docker.pkg.dev/gardener-project/public/gardener/etcd-druid -> localhost:5001/europe-docker_pkg_dev_gardener-project_public_gardener_etcd-druid:v0.23.0-8-gd2047f4a-dirty
Checking cache...
 - europe-docker.pkg.dev/gardener-project/public/gardener/etcd-druid: Not found. Building
Starting build...
Found [kind-etcd-druid-e2e] context, using local docker daemon.
Building [europe-docker.pkg.dev/gardener-project/public/gardener/etcd-druid]...
Target platforms: [linux/arm64]
Using base gcr.io/distroless/static-debian11:nonroot@sha256:63ebe035fbdd056ed682e6a87b286d07d3f05f12cb46f26b2b44fc10fc4a59ed for github.com/gardener/etcd-druid
Using build config europe-docker.pkg.dev/gardener-project/public/gardener/etcd-druid for github.com/gardener/etcd-druid
Building github.com/gardener/etcd-druid for linux/arm64/v8
Publishing localhost:5001/europe-docker_pkg_dev_gardener-project_public_gardener_etcd-druid:v0.23.0-8-gd2047f4a-dirty
Published localhost:5001/europe-docker_pkg_dev_gardener-project_public_gardener_etcd-druid:v0.23.0-8-gd2047f4a-dirty@sha256:e4fa4b60fa3143d6dd5c3118332bfecef1162071370af880358c9a9de1ef7c8b
Build [europe-docker.pkg.dev/gardener-project/public/gardener/etcd-druid] succeeded
Starting test...
Tags used in deployment:
 - europe-docker.pkg.dev/gardener-project/public/gardener/etcd-druid -> localhost:5001/europe-docker_pkg_dev_gardener-project_public_gardener_etcd-druid:v0.23.0-8-gd2047f4a-dirty@sha256:e4fa4b60fa3143d6dd5c3118332bfecef1162071370af880358c9a9de1ef7c8b
Starting deploy...
Helm release etcd-druid not installed. Installing...
NAME: etcd-druid
LAST DEPLOYED: Thu Nov  7 14:46:32 2024
NAMESPACE: e2e-test
STATUS: deployed
REVISION: 1
TEST SUITE: None
Waiting for deployments to stabilize...
 - e2e-test:deployment/etcd-druid is ready.
Deployments stabilized in 2.106 seconds
You can also run [skaffold run --tail] to get the logs

-------------------
Running e2e tests
-------------------
Running Suite: e2e Suite - /Users/renormalize/go/src/github.com/gardener/etcd-druid/test/e2e
========================================================================================
Random Seed: 1730971000

Will run 8 of 8 specs
Running in parallel across 11 processes
------------------------------
[BeforeSuite] PASSED [0.045 seconds]
[BeforeSuite] 
/Users/renormalize/go/src/github.com/gardener/etcd-druid/test/e2e/suite_test.go:67

  Timeline >>
  {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating namespace","namespace":"shoot"}
  {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating secret","secret":{"name":"ca-etcd-aws","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating secret","secret":{"name":"etcd-backup-aws","namespace":"shoot"}}
  << Timeline
------------------------------
[BeforeSuite] PASSED [0.048 seconds]
[BeforeSuite] 
/Users/renormalize/go/src/github.com/gardener/etcd-druid/test/e2e/suite_test.go:67

  Timeline >>
  {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating namespace","namespace":"shoot"}
  {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating secret","secret":{"name":"ca-etcd-aws","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating secret","secret":{"name":"etcd-backup-aws","namespace":"shoot"}}
  << Timeline
------------------------------
[BeforeSuite] PASSED [0.048 seconds]
[BeforeSuite] 
/Users/renormalize/go/src/github.com/gardener/etcd-druid/test/e2e/suite_test.go:67

  Timeline >>
  {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating namespace","namespace":"shoot"}
  {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating secret","secret":{"name":"ca-etcd-aws","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating secret","secret":{"name":"etcd-backup-aws","namespace":"shoot"}}
  << Timeline
------------------------------
[BeforeSuite] PASSED [0.048 seconds]
[BeforeSuite] 
/Users/renormalize/go/src/github.com/gardener/etcd-druid/test/e2e/suite_test.go:67

  Timeline >>
  {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating namespace","namespace":"shoot"}
  {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating secret","secret":{"name":"ca-etcd-aws","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating secret","secret":{"name":"etcd-backup-aws","namespace":"shoot"}}
  << Timeline
------------------------------
[BeforeSuite] PASSED [0.048 seconds]
[BeforeSuite] 
/Users/renormalize/go/src/github.com/gardener/etcd-druid/test/e2e/suite_test.go:67

  Timeline >>
  {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating namespace","namespace":"shoot"}
  {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating secret","secret":{"name":"ca-etcd-aws","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating secret","secret":{"name":"etcd-backup-aws","namespace":"shoot"}}
  << Timeline
------------------------------
[BeforeSuite] PASSED [0.049 seconds]
[BeforeSuite] 
/Users/renormalize/go/src/github.com/gardener/etcd-druid/test/e2e/suite_test.go:67

  Timeline >>
  {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating namespace","namespace":"shoot"}
  {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating secret","secret":{"name":"ca-etcd-aws","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating secret","secret":{"name":"etcd-backup-aws","namespace":"shoot"}}
  << Timeline
------------------------------
[BeforeSuite] PASSED [0.048 seconds]
[BeforeSuite] 
/Users/renormalize/go/src/github.com/gardener/etcd-druid/test/e2e/suite_test.go:67

  Timeline >>
  {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating namespace","namespace":"shoot"}
  {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating secret","secret":{"name":"ca-etcd-aws","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating secret","secret":{"name":"etcd-backup-aws","namespace":"shoot"}}
  << Timeline
------------------------------
[BeforeSuite] PASSED [0.049 seconds]
[BeforeSuite] 
/Users/renormalize/go/src/github.com/gardener/etcd-druid/test/e2e/suite_test.go:67

  Timeline >>
  {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating namespace","namespace":"shoot"}
  {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating secret","secret":{"name":"ca-etcd-aws","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating secret","secret":{"name":"etcd-backup-aws","namespace":"shoot"}}
  << Timeline
------------------------------
[BeforeSuite] PASSED [0.049 seconds]
[BeforeSuite] 
/Users/renormalize/go/src/github.com/gardener/etcd-druid/test/e2e/suite_test.go:67

  Timeline >>
  {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating namespace","namespace":"shoot"}
  {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating secret","secret":{"name":"ca-etcd-aws","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating secret","secret":{"name":"etcd-backup-aws","namespace":"shoot"}}
  << Timeline
------------------------------
[BeforeSuite] PASSED [0.049 seconds]
[BeforeSuite] 
/Users/renormalize/go/src/github.com/gardener/etcd-druid/test/e2e/suite_test.go:67

  Timeline >>
  {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating namespace","namespace":"shoot"}
  {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating secret","secret":{"name":"ca-etcd-aws","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating secret","secret":{"name":"etcd-backup-aws","namespace":"shoot"}}
  << Timeline
------------------------------
[SynchronizedAfterSuite] PASSED [0.000 seconds]
[SynchronizedAfterSuite] 
/Users/renormalize/go/src/github.com/gardener/etcd-druid/test/e2e/suite_test.go:117
------------------------------
[SynchronizedAfterSuite] PASSED [0.000 seconds]
[SynchronizedAfterSuite] 
/Users/renormalize/go/src/github.com/gardener/etcd-druid/test/e2e/suite_test.go:117
------------------------------
[BeforeSuite] PASSED [0.068 seconds]
[BeforeSuite] 
/Users/renormalize/go/src/github.com/gardener/etcd-druid/test/e2e/suite_test.go:67

  Timeline >>
  {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating namespace","namespace":"shoot"}
  {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating secret","secret":{"name":"ca-etcd-aws","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating secret","secret":{"name":"etcd-server-cert-aws","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating secret","secret":{"name":"etcd-client-tls-aws","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating secret","secret":{"name":"etcd-backup-aws","namespace":"shoot"}}
  << Timeline
------------------------------
[SynchronizedAfterSuite] PASSED [0.001 seconds]
[SynchronizedAfterSuite] 
/Users/renormalize/go/src/github.com/gardener/etcd-druid/test/e2e/suite_test.go:117
------------------------------
• [157.535 seconds]
Etcd Backup when single-node etcd is configured with provider aws Should create, test backup and delete etcd with backup
/Users/renormalize/go/src/github.com/gardener/etcd-druid/test/e2e/etcd_backup_test.go:58

  Timeline >>
  STEP: Purge snapstore @ 11/07/24 14:46:50.621
  STEP: Create etcd @ 11/07/24 14:46:50.634
  {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"Waiting for etcd to become ready","etcd":{"name":"etcd-single-node-aws","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:48:17+05:30","msg":"etcd is ready","etcd":{"name":"etcd-single-node-aws","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:48:17+05:30","msg":"Checking statefulset","etcd":{"name":"etcd-single-node-aws","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:48:17+05:30","msg":"Checking configmap","etcd":{"name":"etcd-single-node-aws","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:48:17+05:30","msg":"Checking client service","etcd":{"name":"etcd-single-node-aws","namespace":"shoot"}}
  STEP: Create debug pod @ 11/07/24 14:48:17.081
  STEP: Check initial snapshot is available @ 11/07/24 14:48:27.123
  STEP: Put keys into etcd @ 11/07/24 14:48:27.177
  {"level":"info","ts":"2024-11-07T14:48:27+05:30","msg":"populating etcd with sequential key-value pairs","fromKey":"foo-1","fromValue":"bar-1","toKey":"foo-10","toValue":"bar-10"}
  {"level":"info","ts":"2024-11-07T14:48:31+05:30","msg":"put key-value successful","key":"foo-1","value":"bar-1"}
  {"level":"info","ts":"2024-11-07T14:48:31+05:30","msg":"put key-value successful","key":"foo-2","value":"bar-2"}
  {"level":"info","ts":"2024-11-07T14:48:32+05:30","msg":"put key-value successful","key":"foo-3","value":"bar-3"}
  {"level":"info","ts":"2024-11-07T14:48:32+05:30","msg":"put key-value successful","key":"foo-4","value":"bar-4"}
  {"level":"info","ts":"2024-11-07T14:48:32+05:30","msg":"put key-value successful","key":"foo-5","value":"bar-5"}
  {"level":"info","ts":"2024-11-07T14:48:33+05:30","msg":"put key-value successful","key":"foo-6","value":"bar-6"}
  {"level":"info","ts":"2024-11-07T14:48:33+05:30","msg":"put key-value successful","key":"foo-7","value":"bar-7"}
  {"level":"info","ts":"2024-11-07T14:48:33+05:30","msg":"put key-value successful","key":"foo-8","value":"bar-8"}
  {"level":"info","ts":"2024-11-07T14:48:33+05:30","msg":"put key-value successful","key":"foo-9","value":"bar-9"}
  {"level":"info","ts":"2024-11-07T14:48:34+05:30","msg":"put key-value successful","key":"foo-10","value":"bar-10"}
  STEP: Check snapshot after putting data into etcd @ 11/07/24 14:48:34.277
  STEP: Trigger on-demand full snapshot @ 11/07/24 14:48:34.361
  STEP: Put additional data into etcd @ 11/07/24 14:48:34.414
  {"level":"info","ts":"2024-11-07T14:48:34+05:30","msg":"populating etcd with sequential key-value pairs","fromKey":"foo-11","fromValue":"bar-11","toKey":"foo-15","toValue":"bar-15"}
  {"level":"info","ts":"2024-11-07T14:48:38+05:30","msg":"put key-value successful","key":"foo-11","value":"bar-11"}
  {"level":"info","ts":"2024-11-07T14:48:38+05:30","msg":"put key-value successful","key":"foo-12","value":"bar-12"}
  {"level":"info","ts":"2024-11-07T14:48:38+05:30","msg":"put key-value successful","key":"foo-13","value":"bar-13"}
  {"level":"info","ts":"2024-11-07T14:48:38+05:30","msg":"put key-value successful","key":"foo-14","value":"bar-14"}
  {"level":"info","ts":"2024-11-07T14:48:39+05:30","msg":"put key-value successful","key":"foo-15","value":"bar-15"}
  STEP: Trigger on-demand delta snapshot @ 11/07/24 14:48:39.286
  STEP: Test cluster restoration by deleting data directory @ 11/07/24 14:48:39.335
  {"level":"info","ts":"2024-11-07T14:48:39+05:30","msg":"waiting for sts to become unready","statefulSetName":"etcd-single-node-aws"}
  {"level":"info","ts":"2024-11-07T14:49:03+05:30","msg":"sts is unready","statefulSetName":"etcd-single-node-aws"}
  {"level":"info","ts":"2024-11-07T14:49:03+05:30","msg":"waiting for sts to become ready again","statefulSetName":"etcd-single-node-aws"}
  {"level":"info","ts":"2024-11-07T14:49:19+05:30","msg":"sts is ready","statefulSetName":"etcd-single-node-aws"}
  {"level":"info","ts":"2024-11-07T14:49:19+05:30","msg":"fetching etcd key-value pairs"}
  {"level":"info","ts":"2024-11-07T14:49:19+05:30","msg":"fetched key-value pair from etcd","key":"foo-1","value":"bar-1"}
  {"level":"info","ts":"2024-11-07T14:49:20+05:30","msg":"fetched key-value pair from etcd","key":"foo-2","value":"bar-2"}
  {"level":"info","ts":"2024-11-07T14:49:20+05:30","msg":"fetched key-value pair from etcd","key":"foo-3","value":"bar-3"}
  {"level":"info","ts":"2024-11-07T14:49:20+05:30","msg":"fetched key-value pair from etcd","key":"foo-4","value":"bar-4"}
  {"level":"info","ts":"2024-11-07T14:49:21+05:30","msg":"fetched key-value pair from etcd","key":"foo-5","value":"bar-5"}
  {"level":"info","ts":"2024-11-07T14:49:21+05:30","msg":"fetched key-value pair from etcd","key":"foo-6","value":"bar-6"}
  {"level":"info","ts":"2024-11-07T14:49:21+05:30","msg":"fetched key-value pair from etcd","key":"foo-7","value":"bar-7"}
  {"level":"info","ts":"2024-11-07T14:49:21+05:30","msg":"fetched key-value pair from etcd","key":"foo-8","value":"bar-8"}
  {"level":"info","ts":"2024-11-07T14:49:22+05:30","msg":"fetched key-value pair from etcd","key":"foo-9","value":"bar-9"}
  {"level":"info","ts":"2024-11-07T14:49:22+05:30","msg":"fetched key-value pair from etcd","key":"foo-10","value":"bar-10"}
  {"level":"info","ts":"2024-11-07T14:49:22+05:30","msg":"fetched key-value pair from etcd","key":"foo-11","value":"bar-11"}
  {"level":"info","ts":"2024-11-07T14:49:23+05:30","msg":"fetched key-value pair from etcd","key":"foo-12","value":"bar-12"}
  {"level":"info","ts":"2024-11-07T14:49:23+05:30","msg":"fetched key-value pair from etcd","key":"foo-13","value":"bar-13"}
  {"level":"info","ts":"2024-11-07T14:49:23+05:30","msg":"fetched key-value pair from etcd","key":"foo-14","value":"bar-14"}
  {"level":"info","ts":"2024-11-07T14:49:24+05:30","msg":"fetched key-value pair from etcd","key":"foo-15","value":"bar-15"}
  STEP: Deleting debug pod @ 11/07/24 14:49:24.12
  STEP: Deleting etcd @ 11/07/24 14:49:24.125
  {"level":"info","ts":"2024-11-07T14:49:24+05:30","msg":"Checking if etcd is gone","etcd":{"name":"etcd-single-node-aws","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:49:28+05:30","msg":"Checking if statefulset is gone","etcd":{"name":"etcd-single-node-aws","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:49:28+05:30","msg":"Checking if configmap is gone","etcd":{"name":"etcd-single-node-aws","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:49:28+05:30","msg":"Checking client service is gone","etcd":{"name":"etcd-single-node-aws","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:49:28+05:30","msg":"Deleting PVCs"}
  << Timeline
------------------------------
[SynchronizedAfterSuite] PASSED [0.000 seconds]
[SynchronizedAfterSuite] 
/Users/renormalize/go/src/github.com/gardener/etcd-druid/test/e2e/suite_test.go:117
------------------------------
• [164.388 seconds]
Etcd Compaction when compaction is enabled for single-node etcd with provider aws should test compaction on backup
/Users/renormalize/go/src/github.com/gardener/etcd-druid/test/e2e/etcd_compaction_test.go:53

  Timeline >>
  STEP: Purge snapstore @ 11/07/24 14:46:50.621
  STEP: Create etcd @ 11/07/24 14:46:50.639
  {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"Waiting for etcd to become ready","etcd":{"name":"etcd-compaction-aws","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:48:35+05:30","msg":"etcd is ready","etcd":{"name":"etcd-compaction-aws","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:48:35+05:30","msg":"Checking statefulset","etcd":{"name":"etcd-compaction-aws","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:48:35+05:30","msg":"Checking configmap","etcd":{"name":"etcd-compaction-aws","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:48:35+05:30","msg":"Checking client service","etcd":{"name":"etcd-compaction-aws","namespace":"shoot"}}
  STEP: Create debug pod @ 11/07/24 14:48:35.169
  STEP: Check initial snapshot is available @ 11/07/24 14:48:39.181
  STEP: Put keys into etcd @ 11/07/24 14:48:39.23
  {"level":"info","ts":"2024-11-07T14:48:39+05:30","msg":"populating etcd with sequential key-value pairs","fromKey":"foo-1","fromValue":"bar-1","toKey":"foo-10","toValue":"bar-10"}
  {"level":"info","ts":"2024-11-07T14:48:42+05:30","msg":"put key-value successful","key":"foo-1","value":"bar-1"}
  {"level":"info","ts":"2024-11-07T14:48:43+05:30","msg":"put key-value successful","key":"foo-2","value":"bar-2"}
  {"level":"info","ts":"2024-11-07T14:48:43+05:30","msg":"put key-value successful","key":"foo-3","value":"bar-3"}
  {"level":"info","ts":"2024-11-07T14:48:43+05:30","msg":"put key-value successful","key":"foo-4","value":"bar-4"}
  {"level":"info","ts":"2024-11-07T14:48:44+05:30","msg":"put key-value successful","key":"foo-5","value":"bar-5"}
  {"level":"info","ts":"2024-11-07T14:48:44+05:30","msg":"put key-value successful","key":"foo-6","value":"bar-6"}
  {"level":"info","ts":"2024-11-07T14:48:44+05:30","msg":"put key-value successful","key":"foo-7","value":"bar-7"}
  {"level":"info","ts":"2024-11-07T14:48:45+05:30","msg":"put key-value successful","key":"foo-8","value":"bar-8"}
  {"level":"info","ts":"2024-11-07T14:48:45+05:30","msg":"put key-value successful","key":"foo-9","value":"bar-9"}
  {"level":"info","ts":"2024-11-07T14:48:45+05:30","msg":"put key-value successful","key":"foo-10","value":"bar-10"}
  STEP: Check snapshot after putting data into etcd @ 11/07/24 14:48:45.763
  STEP: Put additional data into etcd @ 11/07/24 14:48:45.85
  {"level":"info","ts":"2024-11-07T14:48:45+05:30","msg":"populating etcd with sequential key-value pairs","fromKey":"foo-11","fromValue":"bar-11","toKey":"foo-15","toValue":"bar-15"}
  {"level":"info","ts":"2024-11-07T14:48:49+05:30","msg":"put key-value successful","key":"foo-11","value":"bar-11"}
  {"level":"info","ts":"2024-11-07T14:48:49+05:30","msg":"put key-value successful","key":"foo-12","value":"bar-12"}
  {"level":"info","ts":"2024-11-07T14:48:50+05:30","msg":"put key-value successful","key":"foo-13","value":"bar-13"}
  {"level":"info","ts":"2024-11-07T14:48:50+05:30","msg":"put key-value successful","key":"foo-14","value":"bar-14"}
  {"level":"info","ts":"2024-11-07T14:48:50+05:30","msg":"put key-value successful","key":"foo-15","value":"bar-15"}
  STEP: Trigger on-demand delta snapshot @ 11/07/24 14:48:50.694
  {"level":"info","ts":"2024-11-07T14:48:50+05:30","msg":"waiting for compaction job to become successful"}
  {"level":"info","ts":"2024-11-07T14:49:25+05:30","msg":"compaction job is successful"}
  STEP: Put additional data into etcd @ 11/07/24 14:49:25.932
  {"level":"info","ts":"2024-11-07T14:49:25+05:30","msg":"populating etcd with sequential key-value pairs","fromKey":"foo-16","fromValue":"bar-16","toKey":"foo-20","toValue":"bar-20"}
  {"level":"info","ts":"2024-11-07T14:49:29+05:30","msg":"put key-value successful","key":"foo-16","value":"bar-16"}
  {"level":"info","ts":"2024-11-07T14:49:29+05:30","msg":"put key-value successful","key":"foo-17","value":"bar-17"}
  {"level":"info","ts":"2024-11-07T14:49:30+05:30","msg":"put key-value successful","key":"foo-18","value":"bar-18"}
  {"level":"info","ts":"2024-11-07T14:49:30+05:30","msg":"put key-value successful","key":"foo-19","value":"bar-19"}
  {"level":"info","ts":"2024-11-07T14:49:30+05:30","msg":"put key-value successful","key":"foo-20","value":"bar-20"}
  STEP: Trigger next on-demand delta snapshot @ 11/07/24 14:49:30.868
  STEP: Verify that there are new delta snapshots as compaction is not triggered yet because delta events have not reached next 15 revision @ 11/07/24 14:49:30.914
  STEP: Deleting debug pod @ 11/07/24 14:49:30.957
  STEP: Deleting etcd @ 11/07/24 14:49:30.96
  {"level":"info","ts":"2024-11-07T14:49:30+05:30","msg":"Checking if etcd is gone","etcd":{"name":"etcd-compaction-aws","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:49:34+05:30","msg":"Checking if statefulset is gone","etcd":{"name":"etcd-compaction-aws","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:49:34+05:30","msg":"Checking if configmap is gone","etcd":{"name":"etcd-compaction-aws","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:49:34+05:30","msg":"Checking client service is gone","etcd":{"name":"etcd-compaction-aws","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:49:34+05:30","msg":"Deleting PVCs"}
  << Timeline
------------------------------
• [229.038 seconds]
Etcd Single-node etcd configuration Scaling up from single-node to multi-node without TLS should scale a single-node etcd (TLS not enabled for peerUrl) to a multi-node etcd cluster (TLS not enabled for peerUrl)
/Users/renormalize/go/src/github.com/gardener/etcd-druid/test/e2e/etcd_multi_node_test.go:138

  Timeline >>
  STEP: Purge snapstore @ 11/07/24 14:46:50.62
  STEP: Creating a single-node etcd @ 11/07/24 14:46:50.636
  {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"Waiting for etcd to become ready","etcd-multi-node":{"name":"scale-up-non-tls","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:48:33+05:30","msg":"etcd is ready","etcd-multi-node":{"name":"scale-up-non-tls","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:48:33+05:30","msg":"Checking statefulset","etcd-multi-node":{"name":"scale-up-non-tls","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:48:33+05:30","msg":"Checking configmap","etcd-multi-node":{"name":"scale-up-non-tls","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:48:33+05:30","msg":"Checking client service","etcd-multi-node":{"name":"scale-up-non-tls","namespace":"shoot"}}
  STEP: Scaling up a healthy cluster (from 1 to 3 replicas) @ 11/07/24 14:48:33.137
  {"level":"info","ts":"2024-11-07T14:50:11+05:30","msg":"Waiting for etcd to become ready","etcd-multi-node":{"name":"scale-up-non-tls","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:50:35+05:30","msg":"etcd is ready","etcd-multi-node":{"name":"scale-up-non-tls","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:50:35+05:30","msg":"Checking statefulset","etcd-multi-node":{"name":"scale-up-non-tls","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:50:35+05:30","msg":"Checking configmap","etcd-multi-node":{"name":"scale-up-non-tls","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:50:35+05:30","msg":"Checking client service","etcd-multi-node":{"name":"scale-up-non-tls","namespace":"shoot"}}
  STEP: Deleting etcd @ 11/07/24 14:50:35.622
  {"level":"info","ts":"2024-11-07T14:50:35+05:30","msg":"Checking if etcd is gone","etcd-multi-node":{"name":"scale-up-non-tls","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:50:39+05:30","msg":"Checking if statefulset is gone","etcd-multi-node":{"name":"scale-up-non-tls","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:50:39+05:30","msg":"Checking if configmap is gone","etcd-multi-node":{"name":"scale-up-non-tls","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:50:39+05:30","msg":"Checking client service is gone","etcd-multi-node":{"name":"scale-up-non-tls","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:50:39+05:30","msg":"Deleting PVCs"}
  << Timeline
------------------------------
[SynchronizedAfterSuite] PASSED [0.000 seconds]
[SynchronizedAfterSuite] 
/Users/renormalize/go/src/github.com/gardener/etcd-druid/test/e2e/suite_test.go:117
------------------------------
• [261.226 seconds]
Etcd Single-node etcd configuration Scaling down from single-node to zero and back up without TLS executes scaling down to 0 and back up to 3 replicas without TLS
/Users/renormalize/go/src/github.com/gardener/etcd-druid/test/e2e/etcd_multi_node_test.go:189

  Timeline >>
  STEP: Purge snapstore @ 11/07/24 14:46:50.62
  STEP: Creating a single-node etcd @ 11/07/24 14:46:50.631
  {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"Waiting for etcd to become ready","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:48:27+05:30","msg":"etcd is ready","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:48:27+05:30","msg":"Checking statefulset","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:48:27+05:30","msg":"Checking configmap","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:48:27+05:30","msg":"Checking client service","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}}
  STEP: Scaling down a healthy cluster (from 1 to 0 replica) @ 11/07/24 14:48:27.134
  {"level":"info","ts":"2024-11-07T14:48:29+05:30","msg":"Waiting for etcd to become ready","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:48:31+05:30","msg":"etcd is ready","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:48:31+05:30","msg":"Checking statefulset","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:48:31+05:30","msg":"Checking configmap","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:48:31+05:30","msg":"Checking client service","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}}
  STEP: Scaling up cluster (from 0 to 1 replica) @ 11/07/24 14:48:31.161
  {"level":"info","ts":"2024-11-07T14:48:33+05:30","msg":"Waiting for etcd to become ready","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:48:57+05:30","msg":"etcd is ready","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:48:57+05:30","msg":"Checking statefulset","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:48:57+05:30","msg":"Checking configmap","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:48:57+05:30","msg":"Checking client service","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}}
  STEP: Scaling up a healthy cluster (from 1 to 3 replica) @ 11/07/24 14:48:57.244
  {"level":"info","ts":"2024-11-07T14:50:35+05:30","msg":"Waiting for etcd to become ready","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:50:57+05:30","msg":"etcd is ready","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:50:57+05:30","msg":"Checking statefulset","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:50:57+05:30","msg":"Checking configmap","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:50:57+05:30","msg":"Checking client service","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}}
  STEP: Deleting etcd @ 11/07/24 14:50:57.734
  {"level":"info","ts":"2024-11-07T14:50:57+05:30","msg":"Checking if etcd is gone","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:51:11+05:30","msg":"Checking if statefulset is gone","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:51:11+05:30","msg":"Checking if configmap is gone","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:51:11+05:30","msg":"Checking client service is gone","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:51:11+05:30","msg":"Deleting PVCs"}
  << Timeline
------------------------------
[SynchronizedAfterSuite] PASSED [0.000 seconds]
[SynchronizedAfterSuite] 
/Users/renormalize/go/src/github.com/gardener/etcd-druid/test/e2e/suite_test.go:117
------------------------------
• [373.654 seconds]
Etcd when multi-node is configured should perform etcd operations excluding zero downtime maintenance
/Users/renormalize/go/src/github.com/gardener/etcd-druid/test/e2e/etcd_multi_node_test.go:51

  Timeline >>
  STEP: Purge snapstore @ 11/07/24 14:46:50.616
  STEP: Creating etcd @ 11/07/24 14:46:50.63
  {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"Waiting for etcd to become ready","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:50:07+05:30","msg":"etcd is ready","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:50:07+05:30","msg":"Checking statefulset","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:50:07+05:30","msg":"Checking configmap","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:50:07+05:30","msg":"Checking client service","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  STEP: Hibernating etcd (Scaling down from 3 replicas to 0) @ 11/07/24 14:50:07.485
  {"level":"info","ts":"2024-11-07T14:50:07+05:30","msg":"Waiting for statefulset spec to reflect change in replicas to 0","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:50:09+05:30","msg":"Checking etcd","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:50:17+05:30","msg":"Checking statefulset","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:50:17+05:30","msg":"etcd is hibernated","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  STEP: Waking up etcd (Scaling up from 0 to 3 replicas) @ 11/07/24 14:50:17.532
  {"level":"info","ts":"2024-11-07T14:50:19+05:30","msg":"Waiting for etcd to become ready","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:51:47+05:30","msg":"etcd is ready","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:51:47+05:30","msg":"Checking statefulset","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:51:47+05:30","msg":"Checking configmap","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:51:47+05:30","msg":"Checking client service","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  STEP: Member restart with data-dir/pvc intact @ 11/07/24 14:51:47.927
  {"level":"info","ts":"2024-11-07T14:51:47+05:30","msg":"Deleting one member pod","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:51:47+05:30","msg":"waiting for sts to become unready","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:51:49+05:30","msg":"sts is unready","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:51:49+05:30","msg":"Waiting for at least one etcd member to become unready","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:51:53+05:30","msg":"at least one etcd member is unready","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:51:53+05:30","msg":"Waiting for etcd to become ready","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:52:14+05:30","msg":"etcd is ready","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:52:14+05:30","msg":"Checking statefulset","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:52:14+05:30","msg":"Checking configmap","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:52:14+05:30","msg":"Checking client service","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  STEP: Single member restoration @ 11/07/24 14:52:14.023
  {"level":"info","ts":"2024-11-07T14:52:14+05:30","msg":"Creating debug pod","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:52:18+05:30","msg":"Deleting member directory of one member pod","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:52:18+05:30","msg":"waiting for sts to become unready","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:52:34+05:30","msg":"sts is unready","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:52:34+05:30","msg":"Waiting for at least one etcd member to become unready","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:52:38+05:30","msg":"at least one etcd member is unready","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:52:38+05:30","msg":"Waiting for etcd to become ready","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:52:54+05:30","msg":"etcd is ready","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:52:54+05:30","msg":"Checking statefulset","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:52:54+05:30","msg":"Checking configmap","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:52:54+05:30","msg":"Checking client service","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  STEP: Deleting debug pod @ 11/07/24 14:52:54.225
  STEP: Deleting etcd @ 11/07/24 14:52:54.227
  {"level":"info","ts":"2024-11-07T14:52:54+05:30","msg":"Checking if etcd is gone","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:53:04+05:30","msg":"Checking if statefulset is gone","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:53:04+05:30","msg":"Checking if configmap is gone","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:53:04+05:30","msg":"Checking client service is gone","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:53:04+05:30","msg":"Deleting PVCs"}
  << Timeline
------------------------------
[SynchronizedAfterSuite] PASSED [0.000 seconds]
[SynchronizedAfterSuite] 
/Users/renormalize/go/src/github.com/gardener/etcd-druid/test/e2e/suite_test.go:117
------------------------------
• [379.657 seconds]
Etcd Single-node etcd configuration Scaling up from single-node to multi-node with TLS executes scaling up with TLS
/Users/renormalize/go/src/github.com/gardener/etcd-druid/test/e2e/etcd_multi_node_test.go:164

  Timeline >>
  STEP: Purge snapstore @ 11/07/24 14:46:50.62
  STEP: Creating a single-node etcd @ 11/07/24 14:46:50.632
  {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"Waiting for etcd to become ready","etcd-multi-node":{"name":"scale-up-tls","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:48:27+05:30","msg":"etcd is ready","etcd-multi-node":{"name":"scale-up-tls","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:48:27+05:30","msg":"Checking statefulset","etcd-multi-node":{"name":"scale-up-tls","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:48:27+05:30","msg":"Checking configmap","etcd-multi-node":{"name":"scale-up-tls","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:48:27+05:30","msg":"Checking client service","etcd-multi-node":{"name":"scale-up-tls","namespace":"shoot"}}
  STEP: Scaling up a healthy cluster (from 1 to 3 replicas) with TLS enabled for peerUrl @ 11/07/24 14:48:27.134
  {"level":"info","ts":"2024-11-07T14:51:13+05:30","msg":"Waiting for etcd to become ready","etcd-multi-node":{"name":"scale-up-tls","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:52:54+05:30","msg":"etcd is ready","etcd-multi-node":{"name":"scale-up-tls","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:52:54+05:30","msg":"Checking statefulset","etcd-multi-node":{"name":"scale-up-tls","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:52:54+05:30","msg":"Checking configmap","etcd-multi-node":{"name":"scale-up-tls","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:52:54+05:30","msg":"Checking client service","etcd-multi-node":{"name":"scale-up-tls","namespace":"shoot"}}
  STEP: Deleting etcd @ 11/07/24 14:52:54.204
  {"level":"info","ts":"2024-11-07T14:52:54+05:30","msg":"Checking if etcd is gone","etcd-multi-node":{"name":"scale-up-tls","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:53:10+05:30","msg":"Checking if statefulset is gone","etcd-multi-node":{"name":"scale-up-tls","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:53:10+05:30","msg":"Checking if configmap is gone","etcd-multi-node":{"name":"scale-up-tls","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:53:10+05:30","msg":"Checking client service is gone","etcd-multi-node":{"name":"scale-up-tls","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:53:10+05:30","msg":"Deleting PVCs"}
  << Timeline
------------------------------
[SynchronizedAfterSuite] PASSED [0.000 seconds]
[SynchronizedAfterSuite] 
/Users/renormalize/go/src/github.com/gardener/etcd-druid/test/e2e/suite_test.go:117
------------------------------
• [415.862 seconds]
Etcd Single-node etcd configuration Scaling down from single-node to zero and back up with TLS should scale down a single-node etcd to 0 replica, then scale up from 0->1 replica and then from 1->3 replicas with TLS enabled for cluster peerUrl
/Users/renormalize/go/src/github.com/gardener/etcd-druid/test/e2e/etcd_multi_node_test.go:224

  Timeline >>
  STEP: Purge snapstore @ 11/07/24 14:46:50.621
  STEP: Creating a single-node etcd @ 11/07/24 14:46:50.636
  {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"Waiting for etcd to become ready","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:48:35+05:30","msg":"etcd is ready","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:48:35+05:30","msg":"Checking statefulset","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:48:35+05:30","msg":"Checking configmap","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:48:35+05:30","msg":"Checking client service","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}}
  STEP: Scaling down a healthy cluster (from 1 to 0 replica) @ 11/07/24 14:48:35.156
  {"level":"info","ts":"2024-11-07T14:48:37+05:30","msg":"Waiting for etcd to become ready","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:48:39+05:30","msg":"etcd is ready","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:48:39+05:30","msg":"Checking statefulset","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:48:39+05:30","msg":"Checking configmap","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:48:39+05:30","msg":"Checking client service","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}}
  STEP: Scaling up cluster (from 0 to 1 replica) @ 11/07/24 14:48:39.197
  {"level":"info","ts":"2024-11-07T14:48:41+05:30","msg":"Waiting for etcd to become ready","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:49:05+05:30","msg":"etcd is ready","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:49:05+05:30","msg":"Checking statefulset","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:49:05+05:30","msg":"Checking configmap","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:49:05+05:30","msg":"Checking client service","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}}
  STEP: Scaling up a healthy cluster (from 1 to 3 replica) with TLS enabled for peerUrl @ 11/07/24 14:49:05.294
  {"level":"info","ts":"2024-11-07T14:51:51+05:30","msg":"Waiting for etcd to become ready","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:53:32+05:30","msg":"etcd is ready","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:53:32+05:30","msg":"Checking statefulset","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:53:32+05:30","msg":"Checking configmap","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:53:32+05:30","msg":"Checking client service","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}}
  STEP: Deleting etcd @ 11/07/24 14:53:32.364
  {"level":"info","ts":"2024-11-07T14:53:32+05:30","msg":"Checking if etcd is gone","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:53:46+05:30","msg":"Checking if statefulset is gone","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:53:46+05:30","msg":"Checking if configmap is gone","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:53:46+05:30","msg":"Checking client service is gone","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:53:46+05:30","msg":"Deleting PVCs"}
  << Timeline
------------------------------
[SynchronizedAfterSuite] PASSED [0.000 seconds]
[SynchronizedAfterSuite] 
/Users/renormalize/go/src/github.com/gardener/etcd-druid/test/e2e/suite_test.go:117
------------------------------
• [436.551 seconds]
Etcd when multi-node is configured should perform zero downtime maintenance operation: defragmentation
/Users/renormalize/go/src/github.com/gardener/etcd-druid/test/e2e/etcd_multi_node_test.go:94

  Timeline >>
  STEP: Purge snapstore @ 11/07/24 14:46:50.62
  STEP: Creating etcd @ 11/07/24 14:46:50.636
  {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"Waiting for etcd to become ready","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:50:01+05:30","msg":"etcd is ready","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:50:01+05:30","msg":"Checking statefulset","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:50:01+05:30","msg":"Checking configmap","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:50:01+05:30","msg":"Checking client service","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}}
  STEP: Deploying etcd zero downtime validator job @ 11/07/24 14:50:01.461
  {"level":"info","ts":"2024-11-07T14:50:01+05:30","msg":"Creating job to ensure etcd zero downtime","job":"etcd-zero-down-time-validator-rolling-update"}
  {"level":"info","ts":"2024-11-07T14:50:15+05:30","msg":"Job is ready","job":"etcd-zero-down-time-validator-rolling-update"}
  STEP: Conducting zero downtime rolling updates @ 11/07/24 14:50:15.527
  {"level":"info","ts":"2024-11-07T14:50:57+05:30","msg":"Waiting for etcd to become ready","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:51:17+05:30","msg":"etcd is ready","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:51:17+05:30","msg":"Checking statefulset","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:51:17+05:30","msg":"Checking configmap","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:51:17+05:30","msg":"Checking client service","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:51:17+05:30","msg":"Etcd Cluster is healthy and there is no downtime","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}}
  STEP: Performing zero downtime maintenance operation: defragmentation @ 11/07/24 14:51:17.853
  {"level":"info","ts":"2024-11-07T14:51:17+05:30","msg":"Configuring defragmentation schedule for every 1 minute","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:52:00+05:30","msg":"Waiting for etcd to become ready","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:52:22+05:30","msg":"etcd is ready","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:52:22+05:30","msg":"Checking statefulset","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:52:22+05:30","msg":"Checking configmap","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:52:22+05:30","msg":"Checking client service","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:52:22+05:30","msg":"Etcd Cluster is healthy and there is no downtime","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:52:22+05:30","msg":"Waiting for defragmentation to finish","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:54:01+05:30","msg":"Defragmentation is finished","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:54:01+05:30","msg":"Checking for any Etcd downtime","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:54:01+05:30","msg":"Etcd Cluster is healthy and there is no downtime","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}}
  STEP: Deleting etcd @ 11/07/24 14:54:01.094
  {"level":"info","ts":"2024-11-07T14:54:01+05:30","msg":"Checking if etcd is gone","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:54:07+05:30","msg":"Checking if statefulset is gone","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:54:07+05:30","msg":"Checking if configmap is gone","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:54:07+05:30","msg":"Checking client service is gone","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2024-11-07T14:54:07+05:30","msg":"Deleting PVCs"}
  << Timeline
------------------------------
[SynchronizedAfterSuite] PASSED [0.000 seconds]
[SynchronizedAfterSuite] 
/Users/renormalize/go/src/github.com/gardener/etcd-druid/test/e2e/suite_test.go:117
------------------------------
[SynchronizedAfterSuite] PASSED [316.375 seconds]
[SynchronizedAfterSuite] 
/Users/renormalize/go/src/github.com/gardener/etcd-druid/test/e2e/suite_test.go:117

  Timeline >>
  {"level":"info","ts":"2024-11-07T14:54:07+05:30","msg":"Checking for Etcd resources before deleting namespace","namespace":"shoot","namespace":"shoot"}
  {"level":"info","ts":"2024-11-07T14:54:07+05:30","msg":"No Etcd resources found; proceeding with namespace deletion","namespace":"shoot","namespace":"shoot"}
  << Timeline
------------------------------

Ran 8 of 8 Specs in 480.822 seconds
SUCCESS! -- 8 Passed | 0 Failed | 0 Pending | 0 Skipped


Ginkgo ran 1 suite in 8m10.919516666s
Test Suite Passed
Exporting logs for cluster "etcd-druid-e2e" to:
/tmp/etcd-druid-e2e
Deleting kind cluster...
Deleting cluster "etcd-druid-e2e" ...
Deleted nodes: ["etcd-druid-e2e-control-plane"]
Stopping running container kind-registry...
Removing container kind-registry...

@anveshreddy18
Copy link
Contributor

/assign

anveshreddy18
anveshreddy18 approved these changes Nov 8, 2024
View reviewed changes
Copy link
Contributor

@anveshreddy18 anveshreddy18 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR.
LGTM

@renormalize renormalize merged commit 61bf7a0 into gardener:master Nov 8, 2024
9 checks passed
@gardener-robot gardener-robot added the status/closed Issue is closed (either delivered or triaged) label Nov 8, 2024
@renormalize renormalize modified the milestones: 2024-Q4, v0.31.0 Nov 8, 2024
@renormalize renormalize deleted the etcd-v3.4.34 branch November 15, 2024 09:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@seshachalam-yv seshachalam-yv seshachalam-yv approved these changes

@anveshreddy18 anveshreddy18 anveshreddy18 approved these changes

Assignees

@renormalize renormalize

@anveshreddy18 anveshreddy18

Labels
area/control-plane Control plane related area/dev-productivity Developer productivity related (how to improve development) area/performance Performance (across all domains, such as control plane, networking, storage, etc.) related dependencies Pull requests that update a dependency file kind/cleanup Something that is not needed anymore and can be cleaned up kind/enhancement Enhancement, improvement, extension needs/changes Needs (more) changes needs/ok-to-test Needs approval for testing (check PR in detail before setting this label because PR is run on CI/CD) needs/review Needs review needs/second-opinion Needs second review by someone else platform/all size/xl Size of pull request is huge (see gardener-robot robot/bots/size.py) status/closed Issue is closed (either delivered or triaged)
Projects
None yet
Milestone
v0.31.0
Development

Successfully merging this pull request may close these issues.

[Enhancement] Upgrade go.etcd.io/etcd to v3.4.34
7 participants
@renormalize @seshachalam-yv @anveshreddy18 @gardener-robot-ci-1 @gardener-robot-ci-2 @gardener-robot-ci-3 @gardener-robot